[小ネタ]冗長化したNAT Gatewayを配置したVPCでTransit Gatewayのルーティング設定

[小ネタ]冗長化したNAT Gatewayを配置したVPCでTransit Gatewayのルーティング設定

#AWS Transit Gateway
#NAT Gateway
#AWS
なかはら

なかはら

facebook logohatena logotwitter logo
Clock Icon2021.06.18

この記事は公開されてから1年以上経過しています。情報が古い可能性がありますので、ご注意ください。

はじめに

Outbound用VPCに冗長化したNAT Gateway(以下、NATGW)をTransit Gateway(以下、TGW)経由して通信する環境を構築する際のルーティングを整理してみました。

誤った構成での疎通テスト

構築用のCFnテンプレートを用意しました。これでStackを作成します。

AWSのCLI実行例

aws cloudformation create-stack --stack-name outbound-vpc --template-body file://outbound_vpc.yml


Outbound用VPCテンプレート 
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Network Template."

Resources:
  # -----
  # TGW
  # -----
  TransitGatewayA:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw1
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw1

  # -----
  # VPC1
  # -----
  vpc1:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc1
  # -----
  # VPC1 Internet gateway
  # -----
  vpc1internetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: vpc1-igw
  vpc1attachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref vpc1
      InternetGatewayId: !Ref vpc1internetGateway
  # -----
  # VPC1 Public Subnet
  # -----
  vpc1PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.0.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-a
  vpc1PublicSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.1.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-c
  # -----
  # VPC1 Public RouteTable
  # -----
  vpc1PublicRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-a
  vpc1PublicRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingA2:
    Type: AWS::EC2::Route
    DependsOn: vpc1Aattachment
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetC
      RouteTableId: !Ref vpc1PublicRouteTableA
  # -----
  # VPC1 NATGW
  # -----
  vpc1NatgatewayEIP1:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayA:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP1.AllocationId
      SubnetId: !Ref vpc1PublicSubnetA
      Tags:
        - Key: Name
          Value: vpc1-natgw-a
  vpc1NatgatewayEIP2:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayC:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP2.AllocationId
      SubnetId: !Ref vpc1PublicSubnetC
      Tags:
        - Key: Name
          Value: vpc1-natgw-c
  # -----
  # VPC1 Private Subnet
  # -----
  vpc1PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.10.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-a
  vpc1PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.11.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-c
  # -----
  # VPC1 Private RouteTable
  # -----
  vpc1PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-a
  vpc1PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-c
  vpc1PrivateRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  vpc1PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  # -----
  # VPC1 TGW Subnet
  # -----
  vpc1TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.0/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-a
  vpc1TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.16/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-c
  # -----
  # VPC1 TGW RouteTable
  # -----
  vpc1TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-a
  vpc1TgwRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-c
  vpc1TgwRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1TgwRouteTableC
  vpc1TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetC
      RouteTableId: !Ref vpc1TgwRouteTableC

  # -----
  # VPC2
  # -----
  vpc2:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.1.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc2
  # -----
  # VPC2 Private Subnet
  # -----
  vpc2PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.10.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-a
  vpc2PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.11.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-c
  # -----
  # VPC2 Private RouteTable
  # -----
  vpc2PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-a
  vpc2PrivateRoutingA1:
    Type: AWS::EC2::Route
    DependsOn: vpc2Aattachment
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetC
      RouteTableId: !Ref vpc2PrivateRouteTableA
  # -----
  # VPC2 TGW Subnet
  # -----
  vpc2TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.0/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-a
  vpc2TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.16/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-c
  # -----
  # VPC2 TGW RouteTable
  # -----
  vpc2TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-rtb-a
  vpc2TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetA
      RouteTableId: !Ref vpc2TgwRouteTableA
  vpc2TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetC
      RouteTableId: !Ref vpc2TgwRouteTableA

  # -----
  # TGW VPC ATTATCH
  # -----
  vpc1Aattachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetA
        - !Ref vpc1TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach
  vpc2Aattachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetA
        - !Ref vpc2TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach
  # -----
  # TGW Route
  # -----
  TgwRouteTableA:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayA
  TgwRouteA1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc2Aattachment
  TgwRouteA2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc1Aattachment
  vpc1RouteTableAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1Aattachment
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc2RouteTableAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2Aattachment
      TransitGatewayRouteTableId: !Ref TgwRouteTableA

まずはOutbound用VPC(VPC1)とVPC2を作成し、パブリックサブネット(VPC1のみ)、プライベートサブネット、TGW用サブネットをAZ毎に作成しておきます。 TGW用サブネットではTGW間を通信するためのENIがサブネット毎に一つ作成されます。また、ルートテーブルの制御があるのでベストプラクティスに沿って /28 でサブネットを作成します。

VPC1(構成では10.0.0.0/16のVPC)にはNATGWを冗長化します。 TGWはVPCにアタッチするので TGWが良い感じにルーティングしてくれるんだ と思ってTGWリソースを一つ作成してそれぞれのVPCおよびTGW用サブネットをアタッチしました。
後に説明しますが、「TGWが良い感じにルーティングしてくれるんだ」ココを勘違いしてました。

構成が以下の通りです。

これで疎通テストしてみます。
Az-aのプライベートサブネットでEC2インスタンスを起動してインターネット向け通信をおこないます。 起動するEC2インスタンスはAmazon Linux2でセッションマネージャでアクセスする為にAmazonSSMFullAccess ポリシーがアタッチされたIAMロールを作成してEC2インスタンスにアタッチして以下のユーザーデータを設定して起動します。

#!/bin/bash
cd /tmp
sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent

他のOSでセッションマネージャを設定する場合は、以下のページを参考にしてください。

EC2インスタンスからインターネットにアクセスできることが確認できます。

$ curl -I https://dev.classmethod.jp
HTTP/2 200 
date: Thu, 17 Jun 2021 16:01:09 GMT
content-type: text/html; charset=utf-8
content-length: 167473

この構成でAZ-aのNATGWを削除して障害を発生させます。
以下のNATGWを削除した以下のテンプレートでCFn Stackを更新してください。

AWSCLIの実行例

aws cloudformation create-change-set --stack-name outbound-vpc --change-set-name change --template-body file://outbound_vpc.yml


NATGW1を削除したOutbound用VPCテンプレート 
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Network Template."

Resources:
  # -----
  # TGW
  # -----
  TransitGatewayA:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw1
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw1

  # -----
  # VPC1
  # -----
  vpc1:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc1
  # -----
  # VPC1 Internet gateway
  # -----
  vpc1internetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: vpc1-igw
  vpc1attachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref vpc1
      InternetGatewayId: !Ref vpc1internetGateway
  # -----
  # VPC1 Public Subnet
  # -----
  vpc1PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.0.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-a
  vpc1PublicSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.1.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-c
  # -----
  # VPC1 Public RouteTable
  # -----
  vpc1PublicRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-a
  vpc1PublicRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingA2:
    Type: AWS::EC2::Route
    DependsOn: vpc1Aattachment
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetC
      RouteTableId: !Ref vpc1PublicRouteTableA
  # -----
  # VPC1 NATGW
  # -----
#  vpc1NatgatewayEIP1:
#    Type: AWS::EC2::EIP
#    Properties:
#      Domain: vpc
#  vpc1NatgatewayA:
#    Type: AWS::EC2::NatGateway
#    Properties:
#      AllocationId: !GetAtt vpc1NatgatewayEIP1.AllocationId
#      SubnetId: !Ref vpc1PublicSubnetA
#      Tags:
#        - Key: Name
#          Value: vpc1-natgw-a
  vpc1NatgatewayEIP2:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayC:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP2.AllocationId
      SubnetId: !Ref vpc1PublicSubnetC
      Tags:
        - Key: Name
          Value: vpc1-natgw-c
  # -----
  # VPC1 Private Subnet
  # -----
  vpc1PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.10.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-a
  vpc1PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.11.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-c
  # -----
  # VPC1 Private RouteTable
  # -----
  vpc1PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-a
  vpc1PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-c
#  vpc1PrivateRoutingA1:
#    Type: AWS::EC2::Route
#    Properties:
#      DestinationCidrBlock: 0.0.0.0/0
#      NatGatewayId: !Ref vpc1NatgatewayA
#      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  vpc1PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  # -----
  # VPC1 TGW Subnet
  # -----
  vpc1TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.0/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-a
  vpc1TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.16/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-c
  # -----
  # VPC1 TGW RouteTable
  # -----
  vpc1TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-a
  vpc1TgwRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-c
#  vpc1TgwRoutingA1:
#    Type: AWS::EC2::Route
#    Properties:
#      DestinationCidrBlock: 0.0.0.0/0
#      NatGatewayId: !Ref vpc1NatgatewayA
#      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1TgwRouteTableC
  vpc1TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetC
      RouteTableId: !Ref vpc1TgwRouteTableC

  # -----
  # VPC2
  # -----
  vpc2:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.1.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc2
  # -----
  # VPC2 Private Subnet
  # -----
  vpc2PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.10.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-a
  vpc2PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.11.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-c
  # -----
  # VPC2 Private RouteTable
  # -----
  vpc2PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-a
  vpc2PrivateRoutingA1:
    Type: AWS::EC2::Route
    DependsOn: vpc2Aattachment
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetC
      RouteTableId: !Ref vpc2PrivateRouteTableA
  # -----
  # VPC2 TGW Subnet
  # -----
  vpc2TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.0/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-a
  vpc2TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.16/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-c
  # -----
  # VPC2 TGW RouteTable
  # -----
  vpc2TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-rtb-a
  vpc2TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetA
      RouteTableId: !Ref vpc2TgwRouteTableA
  vpc2TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetC
      RouteTableId: !Ref vpc2TgwRouteTableA

  # -----
  # TGW VPC ATTATCH
  # -----
  vpc1Aattachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetA
        - !Ref vpc1TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach
  vpc2Aattachment:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetA
        - !Ref vpc2TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach
  # -----
  # TGW Route
  # -----
  TgwRouteTableA:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayA
  TgwRouteA1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc2Aattachment
  TgwRouteA2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc1Aattachment
  vpc1RouteTableAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1Aattachment
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc2RouteTableAssociation:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2Aattachment
      TransitGatewayRouteTableId: !Ref TgwRouteTableA

EC2インスタンスが操作できなくなりました。AZ-cを経由してルーティングされると思っていたのですが、通信できません。

原因は単純で、 TGWのVPC間通信は同一AZのTGW用ENIを介して通信する のでEC2インスタンスからセッションマネージャで必要なAWSサービス(インターネット経由)に接続できなくなってしまった為、セッションが切られてしまいました。

TGWのサービス別資料にも書いてありますね。(反省)

正しい構成での疎通テスト

今度は正しいルーティングの構成です。 作成済みStackを削除した後、Stackを再作成してみます。

AWSのCLI実行例

aws cloudformation create-stack --stack-name outbound-vpc --template-body file://outbound_vpc_ha.yml


正しくルーティングされたOutbound用VPCテンプレート 
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Network Template."

Resources:
  # -----
  # TGW
  # -----
  TransitGatewayA:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw1
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw1
  TransitGatewayC:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw2
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw2

  # -----
  # VPC1
  # -----
  vpc1:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc1
  # -----
  # VPC1 Internet gateway
  # -----
  vpc1internetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: vpc1-igw
  vpc1attachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref vpc1
      InternetGatewayId: !Ref vpc1internetGateway
  # -----
  # VPC1 Public Subnet
  # -----
  vpc1PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.0.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-a
  vpc1PublicSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.1.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-c
  # -----
  # VPC1 Public RouteTable
  # -----
  vpc1PublicRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-a
  vpc1PublicRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-c
  vpc1PublicRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingA2:
    Type: AWS::EC2::Route
    DependsOn: vpc1AattachmentA
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableC
  vpc1PublicRoutingC2:
    Type: AWS::EC2::Route
    DependsOn: vpc1AattachmentC
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayC
      RouteTableId: !Ref vpc1PublicRouteTableC
  vpc1PublicRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetC
      RouteTableId: !Ref vpc1PublicRouteTableC
  # -----
  # VPC1 NATGW
  # -----
  vpc1NatgatewayEIP1:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayA:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP1.AllocationId
      SubnetId: !Ref vpc1PublicSubnetA
      Tags:
        - Key: Name
          Value: vpc1-natgw-a
  vpc1NatgatewayEIP2:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayC:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP2.AllocationId
      SubnetId: !Ref vpc1PublicSubnetC
      Tags:
        - Key: Name
          Value: vpc1-natgw-c
  # -----
  # VPC1 Private Subnet
  # -----
  vpc1PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.10.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-a
  vpc1PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.11.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-c
  # -----
  # VPC1 Private RouteTable
  # -----
  vpc1PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-a
  vpc1PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-c
  vpc1PrivateRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  vpc1PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  # -----
  # VPC1 TGW Subnet
  # -----
  vpc1TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.0/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-a
  vpc1TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.16/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-c
  # -----
  # VPC1 TGW RouteTable
  # -----
  vpc1TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-a
  vpc1TgwRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-c
  vpc1TgwRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1TgwRouteTableC
  vpc1TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetC
      RouteTableId: !Ref vpc1TgwRouteTableC

  # -----
  # VPC2
  # -----
  vpc2:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.1.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc2
  # -----
  # VPC2 Private Subnet
  # -----
  vpc2PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.10.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-a
  vpc2PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.11.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-c
  # -----
  # VPC2 Private RouteTable
  # -----
  vpc2PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-a
  vpc2PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-c
  vpc2PrivateRoutingA1:
    Type: AWS::EC2::Route
    DependsOn: vpc2AattachmentA
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRoutingC1:
    Type: AWS::EC2::Route
    DependsOn: vpc2AattachmentC
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayC
      RouteTableId: !Ref vpc2PrivateRouteTableC
  vpc2PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetC
      RouteTableId: !Ref vpc2PrivateRouteTableC
  # -----
  # VPC2 TGW Subnet
  # -----
  vpc2TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.0/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-a
  vpc2TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.16/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-c
  # -----
  # VPC2 TGW RouteTable
  # -----
  vpc2TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-rtb-a
  vpc2TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetA
      RouteTableId: !Ref vpc2TgwRouteTableA
  vpc2TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetC
      RouteTableId: !Ref vpc2TgwRouteTableA

  # -----
  # TGW VPC ATTATCH
  # -----
  vpc1AattachmentA:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetA
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach-a
  vpc2AattachmentA:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetA
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach-a
  vpc1AattachmentC:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayC
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach-c
  vpc2AattachmentC:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayC
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach-c
  # -----
  # TGW Route
  # -----
  TgwRouteTableA:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayA
  TgwRouteTableC:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayC
  TgwRouteA1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc2AattachmentA
  TgwRouteA2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc1AattachmentA
  TgwRouteC1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
      TransitGatewayAttachmentId: !Ref vpc2AattachmentC
  TgwRouteC2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
      TransitGatewayAttachmentId: !Ref vpc1AattachmentC
  vpc1RouteTableAssociationA:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1AattachmentA
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc2RouteTableAssociationA:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2AattachmentA
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc1RouteTableAssociationC:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1AattachmentC
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
  vpc2RouteTableAssociationC:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2AattachmentC
      TransitGatewayRouteTableId: !Ref TgwRouteTableC

誤った構成との違いは、3点です。

  1. TGWリソースを2つ作成し、AZ毎にOutbound通信できるようにルーティングを設定
    • TGW1のサブネットはTGW用サブネット1
    • TGW2のサブネットはTGW用サブネット2
  2. AZ毎のNATGWがVPC2に戻るためのターゲットをルートテーブルで設定
    • AZ-aのルートテーブルのVPC2へのターゲットはTGW1
    • AZ-cのルートテーブルのVPC2へのターゲットはTGW2
  3. VPC2のプライベートサブネット毎のルートテーブルで異なるTGWを設定
    • AZ-aのルートテーブルのターゲットはTGW1
    • AZ-cのルートテーブルのターゲットはTGW2

この構成でAZ-aのNATGWを削除して障害を発生させます。
以下のNATGWを削除した以下のテンプレートでCFn Stackを更新してください。

AWSCLIの実行例

aws cloudformation create-change-set --stack-name outbound-vpc --change-set-name change --template-body file://outbound_vpc.yml


NATGW1を削除したOutbound用VPCテンプレート 
---
AWSTemplateFormatVersion: "2010-09-09"
Description: "Network Template."

Resources:
  # -----
  # TGW
  # -----
  TransitGatewayA:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw1
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw1
  TransitGatewayC:
    Type: AWS::EC2::TransitGateway
    Properties:
      Description: tgw2
      AmazonSideAsn: 64512
      AutoAcceptSharedAttachments: disable
      DefaultRouteTableAssociation: disable
      DefaultRouteTablePropagation: disable
      DnsSupport: enable
      VpnEcmpSupport: enable
      Tags: 
        - Key: Name
          Value: tgw2

  # -----
  # VPC1
  # -----
  vpc1:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc1
  # -----
  # VPC1 Internet gateway
  # -----
  vpc1internetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: vpc1-igw
  vpc1attachIgw:
    Type: "AWS::EC2::VPCGatewayAttachment"
    Properties:
      VpcId: !Ref vpc1
      InternetGatewayId: !Ref vpc1internetGateway
  # -----
  # VPC1 Public Subnet
  # -----
  vpc1PublicSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.0.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-a
  vpc1PublicSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.1.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-subnet-c
  # -----
  # VPC1 Public RouteTable
  # -----
  vpc1PublicRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-a
  vpc1PublicRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-public-rtb-c
  vpc1PublicRoutingA1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingA2:
    Type: AWS::EC2::Route
    DependsOn: vpc1AattachmentA
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref vpc1internetGateway
      RouteTableId: !Ref vpc1PublicRouteTableC
  vpc1PublicRoutingC2:
    Type: AWS::EC2::Route
    DependsOn: vpc1AattachmentC
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayId: !Ref TransitGatewayC
      RouteTableId: !Ref vpc1PublicRouteTableC
  vpc1PublicRouteTableAssociation1:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetA
      RouteTableId: !Ref vpc1PublicRouteTableA
  vpc1PublicRouteTableAssociation2:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PublicSubnetC
      RouteTableId: !Ref vpc1PublicRouteTableC
  # -----
  # VPC1 NATGW
  # -----
#  vpc1NatgatewayEIP1:
#    Type: AWS::EC2::EIP
#    Properties:
#      Domain: vpc
#  vpc1NatgatewayA:
#    Type: AWS::EC2::NatGateway
#    Properties:
#      AllocationId: !GetAtt vpc1NatgatewayEIP1.AllocationId
#      SubnetId: !Ref vpc1PublicSubnetA
#      Tags:
#        - Key: Name
#          Value: vpc1-natgw-a
  vpc1NatgatewayEIP2:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc
  vpc1NatgatewayC:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt vpc1NatgatewayEIP2.AllocationId
      SubnetId: !Ref vpc1PublicSubnetC
      Tags:
        - Key: Name
          Value: vpc1-natgw-c
  # -----
  # VPC1 Private Subnet
  # -----
  vpc1PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.10.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-a
  vpc1PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.11.0/24
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-subnet-c
  # -----
  # VPC1 Private RouteTable
  # -----
  vpc1PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-a
  vpc1PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-private-rtb-c
#  vpc1PrivateRoutingA1:
#    Type: AWS::EC2::Route
#    Properties:
#      DestinationCidrBlock: 0.0.0.0/0
#      NatGatewayId: !Ref vpc1NatgatewayA
#      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  vpc1PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetA
      RouteTableId: !Ref vpc1PrivateRouteTableA
  vpc1PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1PrivateSubnetC
      RouteTableId: !Ref vpc1PrivateRouteTableC
  # -----
  # VPC1 TGW Subnet
  # -----
  vpc1TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.0/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-a
  vpc1TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.0.250.16/28
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-subnet-c
  # -----
  # VPC1 TGW RouteTable
  # -----
  vpc1TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-a
  vpc1TgwRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-tgw-rtb-c
#  vpc1TgwRoutingA1:
#    Type: AWS::EC2::Route
#    Properties:
#      DestinationCidrBlock: 0.0.0.0/0
#      NatGatewayId: !Ref vpc1NatgatewayA
#      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRoutingC1:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref vpc1NatgatewayC
      RouteTableId: !Ref vpc1TgwRouteTableC
  vpc1TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetA
      RouteTableId: !Ref vpc1TgwRouteTableA
  vpc1TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc1TgwSubnetC
      RouteTableId: !Ref vpc1TgwRouteTableC

  # -----
  # VPC2
  # -----
  vpc2:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.1.0.0/16
      EnableDnsHostnames: true
      EnableDnsSupport: true
      Tags:
        - Key: Name
          Value: vpc2
  # -----
  # VPC2 Private Subnet
  # -----
  vpc2PrivateSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.10.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-a
  vpc2PrivateSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.11.0/24
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-subnet-c
  # -----
  # VPC2 Private RouteTable
  # -----
  vpc2PrivateRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-a
  vpc2PrivateRouteTableC:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-private-rtb-c
  vpc2PrivateRoutingA1:
    Type: AWS::EC2::Route
    DependsOn: vpc2AattachmentA
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRoutingC1:
    Type: AWS::EC2::Route
    DependsOn: vpc2AattachmentC
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayId: !Ref TransitGatewayC
      RouteTableId: !Ref vpc2PrivateRouteTableC
  vpc2PrivateRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetA
      RouteTableId: !Ref vpc2PrivateRouteTableA
  vpc2PrivateRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2PrivateSubnetC
      RouteTableId: !Ref vpc2PrivateRouteTableC
  # -----
  # VPC2 TGW Subnet
  # -----
  vpc2TgwSubnetA:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 0
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.0/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-a
  vpc2TgwSubnetC:
    Type: AWS::EC2::Subnet
    Properties:
      AvailabilityZone: !Select
        - 1
        - !GetAZs
          Ref: AWS::Region
      CidrBlock: 10.1.250.16/28
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-subnet-c
  # -----
  # VPC2 TGW RouteTable
  # -----
  vpc2TgwRouteTableA:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-tgw-rtb-a
  vpc2TgwRouteTableAssociation01:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetA
      RouteTableId: !Ref vpc2TgwRouteTableA
  vpc2TgwRouteTableAssociation02:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref vpc2TgwSubnetC
      RouteTableId: !Ref vpc2TgwRouteTableA

  # -----
  # TGW VPC ATTATCH
  # -----
  vpc1AattachmentA:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetA
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach-a
  vpc2AattachmentA:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetA
      TransitGatewayId: !Ref TransitGatewayA
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach-a
  vpc1AattachmentC:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc1TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayC
      VpcId: !Ref vpc1
      Tags:
        - Key: Name
          Value: vpc1-attach-c
  vpc2AattachmentC:
    Type: AWS::EC2::TransitGatewayAttachment
    Properties:
      SubnetIds:
        - !Ref vpc2TgwSubnetC
      TransitGatewayId: !Ref TransitGatewayC
      VpcId: !Ref vpc2
      Tags:
        - Key: Name
          Value: vpc2-attach-c
  # -----
  # TGW Route
  # -----
  TgwRouteTableA:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayA
  TgwRouteTableC:
    Type: AWS::EC2::TransitGatewayRouteTable
    Properties:
      TransitGatewayId: !Ref TransitGatewayC
  TgwRouteA1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc2AattachmentA
  TgwRouteA2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
      TransitGatewayAttachmentId: !Ref vpc1AattachmentA
  TgwRouteC1:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 10.1.0.0/16
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
      TransitGatewayAttachmentId: !Ref vpc2AattachmentC
  TgwRouteC2:
    Type: AWS::EC2::TransitGatewayRoute
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
      TransitGatewayAttachmentId: !Ref vpc1AattachmentC
  vpc1RouteTableAssociationA:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1AattachmentA
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc2RouteTableAssociationA:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2AattachmentA
      TransitGatewayRouteTableId: !Ref TgwRouteTableA
  vpc1RouteTableAssociationC:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc1AattachmentC
      TransitGatewayRouteTableId: !Ref TgwRouteTableC
  vpc2RouteTableAssociationC:
    Type: AWS::EC2::TransitGatewayRouteTableAssociation
    Properties:
      TransitGatewayAttachmentId: !Ref vpc2AattachmentC
      TransitGatewayRouteTableId: !Ref TgwRouteTableC

Az-cにあるEC2インスタンスからインターネット向けにpingしてみました。 無事に疎通できました。通信経路はAZ-cのでルーティングされます。

次に、AZ-aのNATGWを削除して疑似障害を発生させてAz-aにあるEC2インスタンスからインターネット向けにpingしてみました。 通信経路先のNATGWがダウンしているのでタイムアウトします。意図した挙動ですね。

まとめ

TGWのルーティング方法の誤った認識でハマってしまいましたが、無事にOutbound用VPCでNATGWを冗長化することができました。 TGWの考え方、設定方法として誰かの役に立てれば幸いです。

Share this article

facebook logohatena logotwitter logo

関連記事

[登壇レポート] VPC間の接続方法を整理してみた #自治体クラウド勉強会

[登壇レポート] VPC間の接続方法を整理してみた #自治体クラウド勉強会

User avatar
のんピ
2024.11.01
[アップデート] AWS Transit Gateway を経由する通信の送信元をセキュリティグループIDで制御できるようになりました

[アップデート] AWS Transit Gateway を経由する通信の送信元をセキュリティグループIDで制御できるようになりました

User avatar
のんピ
2024.10.01
[登壇レポート]「オンプレミスネットワークとVPCとを接続する際に考慮すべきポイントを考えてみた」というタイトルで登壇しました #自治体クラウド勉強会

[登壇レポート]「オンプレミスネットワークとVPCとを接続する際に考慮すべきポイントを考えてみた」というタイトルで登壇しました #自治体クラウド勉強会

User avatar
のんピ
2024.05.29
AWS Transit Gatewayを使ってAWS Network Firewallを通らないように変更する際にダウンタイムが発生するケースを考えてみた

AWS Transit Gatewayを使ってAWS Network Firewallを通らないように変更する際にダウンタイムが発生するケースを考えてみた

User avatar
のんピ
2024.02.28
Classmethod's white logo
Facebook's logo
X's logo
YouTube's logo

© Classmethod, Inc. All rights reserved.